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I. INTRODUCTION 


A. IMAGE PROCESSING 

Image processing iS concerned with the extraction of 
information from natural images which are acquired from 
image sensors. Information extraction involves the 
detection and recognition of patterns within the image. 

The human eye has an extraordinary pattern recognition 
capability, being able to discern approximately one hundred 
shades of gray. However, the eye 1s not always able to 
extract all the information from an image due to radiometric 
degradation, geometric distortion, and noise introduced 
during recording, transmission, and display of the images. 
These factors can severely limit recognition of patterns or 
objects. One purpose of image processing is to aid the 
human eye in extracting the desired image by removing these 
aqvstort wons.. 

Three methods are available in performing image 
processing operations: digital, optical, and photographic. 
Black and white film can retain a limited range of gray 
level intensities (50 or less), whereas digital computers 
can represent several hundreds or thousands of gray levels. 
(Ref. 1] Optical methods are faster, but do not offer the 


flexibility of digital methods. Flexibility is limited by 


1e2 


such factors as the compromise between computation time and 
the accuracy of the results. [Ref. 2] Computers can be used 
to apply various linear and nonlinear transformations to 
images which cannot be performed optically. Digital 
information extraction techniques can fully exploit the 
statistical nature of digital imagery. These techniques can 
also be used for analysis based on correlation of image data 
with nonimaging data. This includes correlation of remotely 
sensed imagery with nonimaging georeferenced cartographical 
data bases. 

The digital computer, used in numerically oriented 
analysis because of its quantitative character and great 
speed, has become a key tool. Numerically oriented remote 
sensing takes advantage of the computer to emphasize the 
inherently quantitative aspects of the image data, dealing 
with the data rather abstractly as a collection of 
measurements rather than aS an image. Tremendous quantities 
of data are of real value only when the data can be acquired 
and analyzed both rapidly and cost effectively. The growth 
of digital computer technology has enabled the development 
of digital image processing techniques. Because of faster 
and cheaper computational components, large-capacity high- 
density digital data storage devices, and improved display 


technology, the processing, manipulation, and display of 
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large volumes of digital imagery has become possible. 
[Ret.. 3] 
A digital image processing system contains three main 


elements as shown in Figure 1.1 and are defined as follows: 


IMAGE IMAGE IMAGE 


ACQUISITION PROCESSING DISPE awe 





Figure 1.l: Image Processing System 


(1) Image Acquisition. This involves the conversion of a 
scene into a digital representation. This element 
can be performed by a sensor system which is designed 
to view a scene and provide a digital representation 
of it. The acquisition involves the conversion of an 
image from a television signal or film into a digital 
representation. [Ref. 3] An image sensor can be 


characterized by a number of features, including: 


e Signal-tonnomse ratio - a measure of the useful 


information extracted from the sensor's signal; 
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eons waattiatlon in the range of the 
response to light energy; 


e resolution - measure of the smallest detail in 
the image which can be retained by the sensor; 


¢ transfer function - relationship between incoming 
light spatial frequency and output spatial 
frequency; 

¢ integration time - the time in which the sensor 
accumulates charges generated by the incoming 
lta @ hie 


e reading speed - the scanning time for a given 
total spatial resolution and picture size; 


= sopecmual Seis lt tivVrey - the portion of the 


electromagnetic spectrum to be used by the 
sensor. [Ref. 4] 


The sophistication of the acquisition system 
based on the above features and capabilities will 
greatly affect the cost, performance, and reliability 
of the acquisition system. However, no matter how 
sophisticated the system is, certain degradations 
will be introduced into the image. These 
degradations fall into two categories: radiometric 
and geometric distortions. Radiometric degradations 
occur from blurring affects of the imaging system, 
nonlinear amplitude responses, shading, transmission 
noise, atmospheric interference (scattering, 
attenuation, haze), variable surface illumination 
(differences in terrain slope and orientation), and 
change of terrain radiance with viewing angle. 
Geometric distortions can be categorized into three 


ie 


categories: sensor-related such as aberrations in 
the optical system, or nonlinearities and noise in 
the scan-deflection system; sensor-platform related 
caused by attitude and altitude of the sensor; and 
object-related distortions caused by Earth rotation 
and curvature, and terrain relief. [Ref. 1] 

Image Processing. This element provides the digital 
processing of the image or images to produce a 
desired result (Figure 1.2). This processing can 
range from simple enhancement of an image for better 
display of scene detail to more complex processing 
involving several component images. [Ref. 3] Digital 
image processing techniques can be divided into two 
different groups. The first group includes 
quantitative restoration of images to correct for 
degradation and noise, regiStration for overlaying 
and mosaicing, and subjective enhancement of image 
features for interpretation. The second group is 
concerned with the extraction of information from the 
images. This area of analysis includes object 
detection, segmentation of images into 
characteristically different regions, and 
determination of structural relationships among the 
regions. [Ref. 1] Within these two groups fall two 


categories: subjective and quantitative processing. 
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Figure 1.2: Image Processing Steps. [Ref. 1] 
Subjective processing is usually performed in an 
adaptive, interactive, and iterative manner. It is a 
trial and error process, and success is based on the 
ability of the observer to detect information of 
interest in the final or enhanced image. The changes 
achieved in the ‘before' and ‘after' versions of the 
images processed subjectively are often quite 
dramatic, despite the relative computational 
Simplicity of many of the subjective techniques. A 
basic tool which is used in performing subjective 
enhancement and image analysis is the histogram. The 
histogram reveals the distribution of the intensities 
within the image; it is represented graphically as a 
plot of the number of picture elements (pixels) at a 
given intensity, versus the gray level intensity. 


Quantitative techniques are generally performed on an 


iy 


image in a nonadaptive, noninteractive manner. The 
processing method is based on a predefined 
mathematical algorithm, and success in processing is 
based on the correctness of the model. Examples of 
qualitative processing is the removal of radiometric 
and geometric distortions. [Ref. 3] 

This element can reduce some of the requirements 
of the image acquisition system, such as signal to 
noise ratio, dynamic range, transfer function, 
integration time, and reading speed. By reducing 
some of the requirements, the cost of the acquisition 
system can be reduced, and the money saved can be 
used to improve the processing capabilities of the 
complete imaging system. 

(3) Imagen Display The final element provides for 
generation of an output product that can be seen by a 
human observer. This element provides the required 
conversion of digital data into an analog femme 
Processed images can be viewed on a volatile display 
monitor that presents the digitized data in an analog 
form (video signal). The imagery data can be 


recorded on film or other hard copy format. [Ref. 3] 


B. OVERVIEW 
This thesis is concerned with the image processing step 
and specifically with image analysis using segmentation 
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techniques. The segmentation technique used here is called 
the gradient relaxation method. This method utilized an 
iterative probability adjustment process to segment pixels 
into two regions, ‘light' and 'dark'. This method is highly 
dependent on the selection of weighting factors. They 
determine the speed at which the segmentation process 
coverages and to regions pixels will be assigned. Analysis 
is done on noisy infrared images of ships to determine if 
targets can be detected, and/or classified. Detection is 
the ability of the observer to sense that an object of 
interest is in the field of view. Classification is defined 
(in the military sense) as the ability of the observer to 
identify the detected object as to its type. For Army 
operations, classification could be a tank, truck, or 
helicopter. For Naval operations, a large ship, small ship, 
combatant, or merchant vessel would be typical types. At 
different steps of engagement, the need to detect or to 
classify the object will depend upon the situation. 
Chapter II is a survey of contemporary image 
segmentation techniques. These techniques are classified 
into three categories: characteristic feature thresholding, 
edge detection, and region extraction. The specific 
algorithm which is investigated in this thesis is a 
combination of feature thresholding and region extraction, 


using a relaxation or iterative process for the segmentation 
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of the image. Chapter III is a discussion on the gradient 
relaxation algorithm, the particular method used in this 
investigation. This chapter introduces the relaxation 
process and develops the gradient relaxation algorithm. 
This algorithm is applied to several noisy infrared images 
of ships in Chapter IV. An analysis is done on how 
effective the algorithm is in reducing or eliminating noise, 
the ability to detect and classify an object in the field of 
view. The final chapter summarized the results, discusses 
possible applications, implementation of the algorithm, and 


possible future work. 
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IIT. TECHNIQUES USED IN SEGMENTATION 


A. SEGMENTATION BASICS 


A major branch of image processing deals with image 


analysis or scene analysis, where the input is pictorial, 


but the output is a description of the given picture or 


scene. 


The following are examples of image analysis 


problems: 


ee) 


(2) 


eo 


(4) 


The input is text and it is desired to read the text; 
here the description of the input consists of a 
sequence characters. 

The input is a nuclear bubble chamber picture, and it 
is desired to detect and locate certain events (e.g., 
particle collisions); the description consists of a 
set of coordinates and names of event types. 

The input 1s a picture of a miotic cell and the 
output is a 'map' showing the arrangement of the 
chromosomes in a standard order. This output 
requires knowledge of the location and identification 
of the chromosomes. 

The input is an aerial photograph of terrain with the 
desired output being a map showing specific types of 
terrain feature (vegetation, buildings, ships, roads, 


etc.). The construction of this output also requires 
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the location and identification of the desig 
terrain features. [Ref. 5] 

In all of these examples, the description refers to 
specific parts or objects in the picture in terms of their 
properties and the relationships between the objects. Image 
analysis consists of four steps: 

Step 1: Segmentation - This is the partitioning Wome 
image into different regions, each having 
different properties. 

Step 2: Regional descriptions - This procedure is used to 
Characterize the segmented regions by a set of 
descriptors which are not sensitive to such 
variations as changes in size, rotation, or 
translation. These descriptors will bring out 
features which will aid in differentiating 
regions with different attributes. 

Step 3: Relational descriptions - This procedure §deame 
with the organization of these regions into a 
meaningful structure. 

Step 4: Descriptions of similarity - The final Stepedeame 
with the problem of establishing measures of 
similarity between regions in an image. [Ref. 6] 

Image segmentation 1S a critical step in the image 
analysis process because errors in segmentation might 


propagate through the other processes producing an incorrect 
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description of the scene. The question can then be asked, 
what should a good image segmentation be? Regions of an 
image segmentation should be uniform and homogeneous with 
respect to some characteristic such as gray level or 
texture. Region interiors should be simple and contain few 
gaps or holes. Adjacent regions of a segmented image should 
be significantly different in value with respect to the 
characteristic on which the regions are homogeneous. 
Boundaries of each region should be smooth and spatially 
accurate. Achieving these desired properties is difficult 
because precisely uniform and homogeneous regions are 
typically full of small holes and have jagged boundaries. 
Requiring that adjacent regions have a large difference in 
value can cause regions to merge and/or boundaries to be 
Host. All of these effects introduce errors which are 
undesirable. [Ref. 7] 

There is neither a standard approach to nor theory for 
of image segmentation. Segmentation techniques are 
basically ad-hoc and differ in the way each emphasizes one 
or more of the properties discussed previously. In the way 
each strikes a balance between one desired property and 
another property. T. Pavlidis has commented that an image 
segmentation problem is basically one of psychophysical 
perception and therefore not susceptible to a purely 


analytical solution. Any mathematical algorithm must be 
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supplemented with heuristics, involving semantics about the 
Class of images under consideration. Quite often, simple 
heuristics are not enough, and it is essential to introduce 
a priori knowledge about the image. An example of this is 
the dalmatian dog picture (Figure 2.1). Without the priori 
knowledge that a picture consists of a dalmatian dog, most 
human observers would perceive the picture as pure noise. 
However, if the observers are told that the image consists 
of a dalmatian dog, most will identify the dog in the 


picture. [Ref. 8] 





Figure 2.1: This picture is perceived to be random noise. 
Mention ‘dalmatian dog' and that image will be 
seen. [Ref. 8] 


Almost all segmentation techniques are based on either 


the concept of similarity (e.g., characteristic feature 
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clustering) or discontinuity (e.g., edge detection). These 
techniques can be categorized into three areas: (1) 
characteristic feature thresholding or clustering, (2) edge 
detection, and (3) region extraction. [Ref. 9] These 


techniques are discussed in the following sections. 


B. CHARACTERISTIC FEATURE THRESHOLDING 

Characteristic feature or gray-level thresholding is a 
widely used segmentation technique. The general idea is to 
divide the gray scale of a histogram into bands of a similar 
characteristic, e.g., gray level. In general, thresholding 
can be described mathematically as 

S(x,y)=k if T,_, < f(x,y) < Tye k=1l,2,..-,m 

where (x,y) are the x- and y-coordinate of a pixel; S(x,y) 
is the segmented function of (x,y); TyeeeeeT, are the 
threshold values with Ty being the minimum and De being the 
maximum; m is the total number of distinct bands (or labels) 
assigned to the segmented image. The selection of the 
threshold value(s) is not a simple task and can be dependent 
on several factors. If the threshold depends only on 
f(x,y), the gray level, it is called a ‘global threshold'. 
If the value is dependent on f(x,y) and the average gray 
level of the neighborhood around that pixel, it is called a 
‘local threshold’. If the threshold is based on the gray 
level f(x,y), the neighborhood gray level, and the 
coordinates x and y of the pixel, it is called a ‘dynamic 
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threshold'. [Ref. 9] As can be seen, the selection of a 
threshold value is not an easy task, but the selection of 
the threshold is very important. 

There are several methods to select a global threshold. 
Some are based on the gray level histogram, others on local 
properties such as the gradient, or Laplacian of an image, 
and others for an image consisting of an object and 
background where the percent of the object area in the image 
is known. The ‘mode method' is a technique based on the 
gray level histogram where the threshold is selected in the 
valley between the peaks (or modes) of the histogram. MThis 
approach has the advantage that it reduces the probability 
of misclassifying an object point as a background point and 
vice versa. 

However, there are some disadvantages to this technique. 
Spatial information is not used to arrive at the thresholds 
which means there is no assurance that the segmented regions 
are contiguous. The minimum location of the valley may be 
difficult to locate since the valley may be broad and flat. 
Methods have been proposed to sharpen the peaks to more 
clearly define a valley bottom. A. Rosenfeld [Ref. 10] 
proposed an iterative method, called relaxation, to sharpen 
the peaks in enhancing images and their histograms. [Ref. 9] 

A simple example of a bimodal (two peaks) histogram is 


shown in Figure 2.2. The objective is to select T such that 
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band B} contains, as closely as possible, levels associated 
with the background, while Bz contains levels associated 
with the object(s). Each band is assigned a single gray 
level within that band which will best discriminate the 
object from the background. This figure also demonstrates 
the case of a broad and flat valley, where many of the 
pixels in band Bz are not part the object but may be noise, 
therefore part of the background. The iterative method 
mentioned above is a possible solution to enhancing the peak 
at the right creating a truer representation of the object. 
Using the original threshold value, errors will be 
introduced into the scene analysis process, which is 
unacceptable as was stated earlier. This thesis looks at 
the use of the iterative method in selecting a threshold and 


creating a segmented image. 


Mais nn 


Dark Light ee ee 
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Figure 2.2: Histogram thresholding [Ref. 6] 
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C. @5SDGE DETECIIeN 


Edge detection is an image segmentation technique based 


on the discontinuity of gray levels at the boundary between 


different objects. This discontinuity can be any one of 


several geometrical forms: 


(i) 


L2) 


C3) 


An edge ~ The gray level is uniformly consistent in 
each of two adjacent regions, and changes abruptly at 
the border between the regions. 

A line or curve - The gray level of a thin strip in 
the image differs from the two regions on either side 
of the, strip. 

A spot - The gray level is relatively constant except 
at one location in the image. This looks like a 
Spike in a cross-sectional view (Figure 2.3), but 


appears as a spike from all directions. [Ref. 5] 


Figure 2.3: a) Idealized edge cross section. 
b) Perfect 'spike' line. 
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Edge detection schemes consist of three steps: 

(1) The use of a gradient or derivative operator to 
detect locations where the gray level is changing 
rapid ly In the case of digital images, difference 
operators are used instead of derivatives. 

(2) A threshold operation is performed on the gradient in 
order to decide if an edge has been found. The edge 
points are assigned a value greater than the 
background if the gradient is larger than a certain 
threshold. This threshold selection is a key problem 
in noisy images. Too high a threshold does not 
permit the detection of subtle, low-intensity edges. 
A value too low causes noise to be detected as edges. 

(3) Pixels which have been determined to be edges must 
then be linked to form closed curves surrounding the 
regions. [Ref. 11] 

Edge detection is of limited value as an approach to 
segmentation of noisy remotely sensed images. Often the 
edges have gaps at places where the transition between 
regions are not sufficiently abrupt. Additional edges may 
be detected at points that are not part of region 
boundaries, and the detected edges will not form a set of 


closed, connected object boundaries. [Ref. 1] 
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D. REGION EXTRACTION 

Another way of doing segmentation is to divide the image 
into regions. Region extraction techniques can be divided 
into three categories: (1) region merging, (2) region 
splitting, and (3) combination of region merging and 
Spire crng. 

Since the goal of segmentation is to partition an image 
into regions, a direct approach is to attempt a partitioning 
of the image into regions which satisfy a Similarity 
criterion, i1.e., group points into regions. The criteria 
which can be used in extracting objects include region 
homogeneity (in gray level, texture, etc.) and contrast with 
the background, strength of the region's edges, size, shape 
Simplicity, and conformity to a desired texture or shape. 
The advantage of this approach is that it results not only 
in boundary point of regions but also in satisfying a 
Similarity criterion for all points within the regions. In 
order to group points, three fundamental issues must be 
resolved. The first is to determine the number of regions. 
The second is to determine some properties or features which 
distinguish one region from the other regions. The third is 
to specify a suitable similarity criterion which will 
produce a 'meaningful' segmentation. A 'meaningful' 
segmentation is a subjective term and is based on subjective 


methods. [Ref. 6] 
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One method is called region growing. This approach 
starts with very small regions with uniform pixel 
properties. Growth begins by starting with one of these 
regions and merging neighboring regions with it, one at a 
time. The choice of which neighbor to merge will depend on 
both the similarity of the regions (based on gray level, 
texture, etc.) and on the size and shape of the resultant 
merged region. Because of the sequential operations 
involved, the process is slow. 

Another approach is region splitting. This approach 
considers the whole image as a single region, and partitions 
it by repeated splitting. Two Simple approaches of 
subdividing an image are bisection and triangulation. en 
bisection, if the complete image is not homogeneous, it is 
divided into quadrants; if a quadrant is not homogeneous, it 
1s divided again into quadrants; this process continues 
until all of the quadrantS are homogeneous. In 
triangulation, the image is divided into four triangular 
sectors which meet at a point having a gray level farthest 
from the mean; if a triangle is not homogeneous, it is 
divided into four triangles; this continues in a similar 
Manner as in the bisection method. There are two serious 
problems with this technique. The image could be subdivided 


down to the single pixel level, which is probably 
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unacceptable, or the final partition may contain adjacent 
regions with identical characteristics. 

A method which is preferable to either merging or 
splitting is the combination of the two, or the merge-and- 
split method. The general idea is to start with a given 
initial partition; the entire image is a region, each pixel 
or a small block of pixels is a region. Adjacent regions 
are merged if the new region is sufficiently homogeneous, 
and a region will be split if it is not considered to meet a 
homogeneous criteria. [Ref. 5] 

One of the disadvantages of region merging processes is 
their inherently sequential nature. The regions produced 
depend greatly on the order in which regions are merged 
together. Most, if not all region extraction methods rely 
heavily on local information. It is difficult Was 
incorporate global information into an algorithm unless the 
category of pictures to be processed is severely limited. 
All region extraction techniques process pictures in an 
iterative manner which usually involves a large expenditure 
of computational time and memory. 

A method which takes advantage of both parallel and 
sequential methods is called relaxation. 'Parallel' methods 
have the classification decision done at each point 
independently of the decisions at other points. 


'Sequential' methods are those which base their decision on 
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previous decisions. ‘Sequential’ methods are more powerful 
than 'parallel' methods because they learn to better define 
the region classification as they proceed. However, 
'sequential' methods are slower and their results are still 
dependent on the order in which the points are processed. 
[Ref. 9] 

Relaxation is an iterative approach which makes 
probabilistic classification decisions at every pixel in 
parallel at each iteration. It then adjusts these decisions 
at successive iterations based on the decisions made at the 
preceding iteration at the neighboring points. The 
relaxation method 1S conducive to the segmentation problem 
in noisy infrared images. Noise within or near the target 
Will be filtered out due to the sequential process involved 
when the probability classification of the noise pixel is 
adjusted based on its neighbors. The adjustment of the 
pixels to a high probability ('light') or a low probability 
('dark') will enhance the peaks in the histogram, allowing 
for an easy selection of a threshold. The theory for this 
method will be discussed more fully in the next chapter. 
[Ref. 5] In order to evaluate the usefulness of this 
method, experiments are conducted and the results presented 


in Chapter IV. 
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III. SEGMENTATION BY THE GRADIENT RELAXATION METHOD 


Segmentation of an image into regions can be done by 
various methods described in the previous chapter. These 
techniques fall into several categories: region merging, 
region splitting, and a combination of merging and splitting 
as mentioned before. A method which provides for an easy 
selection of a threshold value and combines the advantages 
of sequential and parallel processing techniques is the 
relaxation technique. This chapter will discuss the theory 
behind the relaxation technique and develop the mathematical 
relationships used in the gradient relaxation method, the 


segmentation technique used in this work. 


A. INTRODUCTION TO RELAXATION PROCESSES 

Relaxation, or iterative methods, were originally 
developed as a numerical analysis tool to solve a set of 
Simultaneous equations. In recent years, relaxation methods 
have been applied to image analysis. The classification of 
parts in an image using relaxation techniques was first 
introduced by A. Rosenfeld [Ref. 12] and S. Zucker [Ref. 
boomer. These methods have been applied to histogram 
modification (a peak enhancement scheme), noise cleaning, 
edge and curve detection, curve thinning, angle detection, 
template matching, and region labeling. 
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Image analysis usually involves the discrimination or 
classification of parts within an image. Classification can 
be based on gray level intensity by categorizing points as 
‘light! (object) or 'dark' (background), or vice versa, in 
the segmented infrared images. For edge or non-edge point 
classification, it is based on some local property (e.g., 
the magnitude of the gradient) evaluated at that point. 
Angles on a curve are classified based on the magnitude of 
the curvature of the curve at that point. Classification of 
image points based on these properties is error-prone, 
because noise in the image may cause the local property to 
be misleading. This misclassification can be compounded if 
the classification is done in a 'parallel' fashion, i.e., 
each point is classified without reference to any 
classification decisions of is neighboring points. However, 
if the classification procedure has sequential operations, 
the process takes advantage of previous classification of 
mme neighbor points. This is the basis of the 
classification of objects using relaxation methods. The 
iterative approach has two advantages: (1) classification 
decisions become better informed as the analysis proceeds 
and (2) the method can use fuzzy or probabilistic 
classifications rather than making firm decisions 
immediately as would be the case in a parallel process. 


{Ref. 10] 
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The iterative probabilistic classification method can be 
described in the following manner. A set of objects 
(points, lines, regions, etc.) Ajl,A5,...,Ay are classified 
into a set of classes hj, 149,...,4m- Each obgecesnace 
neighbor relation, i.e., each Aj; has a specified set of Aj's 
as neighbors. Each object Aj is associated with a 
probability vector (Pi1, Pj2,...,Piy) where Pj ise 
estimate of the probability that Aj belongs to a certain 
class k,. The initial probability is based ona 
conventional type of analysis. For example, a point's 
probability is based on its gray level, i.e., proportional 
to the distances of that gray level to the maximum values of 
the gray level range. The next step is to define a measure 
of compatibility between an object A; belonging to Ap, and 
another object Aj belonging to \i,. If there is a high 
compatibility (or similarity) between object Aj; and object 
Aj, l1.e. (Aj, Aszerx), object A; is reinforced by its 
neighbors. Thus its probability is increased. However, if 
the objects are incompatible, the probability remains the 
same or decreases. [Ref. 10] This can be expressed 


mathematically as 
Pip = (Pip) (1 + O5p) 
(2(Pin) (1 + Qin?) 
h 
where qip, a compatibility vector, is defined as 


Nm 
Oih = rZc(i,h, jek) P 4x 
aus 
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where c(i,h,j,k) is the compatibility coefficient between 
object Aj and Ay, with values between [-1,1] (low 
compatibility, high compatibility). [Ref. 14] 

The application of relaxation techniques to segmentation 
involves the classification of pixels into ‘light' and 
'dark' classes. The initial probabilities of each pixel in 
a certain class is based on its gray level, i.e., 
proportional to the distances of the gray level to the 
maximum value of the gray level range. These probabilities 
are iteratively adjusted based on the neighborhood 
probabilities, with ‘light’ reinforcing ‘'light' and ‘'dark' 
reinforcing ‘dark'. This is the basic technique used in the 


algorithm which will be discussed in the following section. 


B. GRADIENT RELAXATION ALGORITHM 
1. Gradient Relaxation Basics 
The segmentation technique which is to be analyzed 
is a region splitting method using a recursive procedure of 
the two-class relaxation technique. The two-class technique 
controls the segmentation process and provides for an 
automatic selection of a threshold. Normally, in the 
application of various segmentation techniques based on 
thresholding, the histogram shows two or more peaks in at 
least one of the spectral features corresponding to various 
homogeneous regions of an image. Very often preprocessing 
is done to alter the histograms and local properties are 


By) 


used to compute the local, global, or dynamic threshold. 
However, if the intensity histogram of the image is 
unimodal, then the application of thresholding techniques 
produces a poor segmentation and does not establish a 
criteria for automatic threshold selection. A unimodal 
distribution is typically obtained when the image consists 
mostly of a large background area with other small but 
Significant objects (or regions) in the image. For example, 
in the case of a complex aerial photographs which may have 
many objects within the scene, the histogram may have only 
one broad peak because the restricted range of intensities 
for the objects is probably covered by the background. 
2. Development of the Gradient Relaxation Algorithm 

In a paper by B. Bhanu and O. Faugeras [{Ref. 15], 
they proposed a gradient relaxation algorithm for the 
segmentation of images having an unimodal distribution. 
This algorithm is based on the use of inconsistency and 
uncertainty to define a global criterion upon the set of 
pixels. Let A, and A2 correspond to two classes, white 
(gray level = 255) and black (gray level = 0), respectively. 


‘Inconsistency' is defined as the difference between the 


probability vector P; = [Pj(Aj1), P(A) | eae 
compatibility vector Q; = [0Q;(A1), O;(Ag)1], of themaeg 
pixel. In other words, what is the discrepancy between what 


every pixel 'thinks' about its own labeling and what its 
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Meigniborsm think sabouc that labeling (0;). ‘'Uncertainty', 


is measured by the entropy function and is defined to be 


1a 1 l 
Smee) ) = ——'P; (Aj, )ln ———— + P3(X\o)1In ———— (et) 
1n2 | P;(A1) Pi (AQ) 


A criterion is defined as 


N 
CUP, Po, 2oe7 PN) Sod Pir; (e221) 


where N is the total number of pixels in the image. The 
goal is to maximize this criterion. The relaxation process 
is specified by choosing a model of interaction between 
pixels and attach to each pixel i the set Vy of its eight 
nearest neighbors. The idea is to make like pixels 
reinforce like pixels by defining a compatibility function 


Cr 


Glisdtm,j,An)=0 mn, for pixel j in Vy for all i 


Cli,Am,jrAm)=1 m=1,2 for pixel j in Vy for all i (3.3) 


where i ranges from 1 to N pixels. 
The compatibility vector, Q;, for the two class case 
is then 
2 
O,(Am)=1/8 f= Z clisAmrjJrAn)P5(An) m=1,2 , i=l,...-n 


jeVy m=l1 
(3.4) 
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Substituting for c, this becomes the mean neighborhood 


probability of the ith pixel for the case being considered, 


Oi (Am)=1/8 £ P4lAm) (3.5) 
JEVG 


The choice of compatibility function in (3.3))wigm 
provide the desired result in the interior of the region, 
but along the edges of a region the pixel label may be 
uncertain because of two different classes of neighbors. 
This may cause distortion at the boundary. 

The maximization of the criterion (3.2) means that a 
local maximum has been sought that is close to the initial 


labeling p, 'o). 


The maximum criterion is achieved by 
aligning the vectors Pj; and Q; while turning them into unit 
vectors. This results in increasing the consistency 
(reducing the difference) and the certainty between the 
vectors Py; and Qj; while turning them into unit vectors. 
This results in increasing the consistency (reducing the 
difference) and the certainty between the vectors Pj and Qj. 
Tt is easily seen from the definition of inconsistency that 
the minimum occurs when P; = Q;. From Figure 3. lye. 
maximum entropy, or high uncertainty, occurs when Pj(Ap) = 
0.5. The maximum certainty occurs when Pj(i\p) = 0.0 or 1.0, 
i.e., P; = [0,1] or [1,0], a undeeveeccew 
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The uncertainty definition clearly shows that the 
initial assignment of probabilities is important because it 
affects the rate of convergence and the final results of the 
relaxation process. The initial probabilities of each pixel 


is defined as 


Pi (Am) = I(i)/G (2G) 


Hip) - 
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Picglzess.i: Entropy Function (Ref. 16] 


where I(i) is the intensity of pixel i in the range 0 < I(i) 
< G, and G is the maximum value of the gray levels. fThis 
definition disregards any a priori knowledge that may be 
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known about an image. However, a priori knowledge can be 


included in the initial probabilities by estimating the 


ratio of white pixels, N,, and the number of black pixels, 


Np. Fhis ratio is 


= Pi (dq) 
1 


<1 Pj (A2) 
i 
(32379) 
= I/(G - I) 


where I is the mean intensity level of the image. By 
knowing this, the distribution of gray levels can be 
modified so as to make the ratio r closer to the true ratio, 


r,- A simple way to do this is to define 


[Toi ) = (PACT) (i (2) = ele (3.8) 


where I, is a desired mean and FACT is a parameter which can 


be chosen to be 


FACT = 1 for 16), e500 


0.7 < FACT < 1.0 £or Tia ae 
Substituting I°(i) in (3.8) into I(i) in (3.6), 
Py (41) = (PACT) (1(i) =" 1)/7G aie Gs (332) 


For the analysis performed in this thesis, the following 


values were used, 
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EAGT = 1.0 


Ga 255 
I. /G = 0.5 
Pi (Az) = (I(i) - 1)/255 + 0.5 (3.10) 


When the first term of (3.10) is greater than 0.5 or 
less than -0.5, then a value of 1.0 or 0.0 will be assigned 
to the probability, respectively. [Ref. 17] 


The gradient of the criterion is obtained from (3.2) 


Meee On Py CADVOUCA2) © 2 P5705 + F PhO 
JeVi keV i 


dC aC 
wie 


ie a (3 dalle 
IPG(Ay) = OPG (AQ) 


Solving for each component of the gradient, we have 


dC 3 
ie aon) ol LZ PS 05 (3.12a) 
OP; (A1) See Je v 7 

dC 3 
a = Of lg + ———— = Ps - 0 (3.12b) 
dPy (A 2) OP4 (X42) JeVvi 


Looking only at (3.12a) and taking the second term only, we 
obtain 
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3 3 3 

Se 2g WISP on :(——»5} . . 

5105 QO, + 2 23.) ee 

ORT CAT om . dP; (2X7) : : ; , 9Pi (Ay) 
JEVy Jevy JeVi * 


The first term is zero because the probabilities of the 
neighbor pixels, P4's, are independent of the probability of 


pixel 1, Pj; therefore 





3 , 3 
—. £ Py-Q5 = 5 r+(——— 93) 
7 3°23 j j 
evs jevi 99; (Aj) 
0 3 
= 2 | P3(d4 955 (1) + PH(AQ) 
JS Al 1 2 Q4(A2) 
JeVi 9P3 (Az) : J 9P5 (A) a 


(3. 138 
Recall that the compatibility function ceo es 


Q35 (Ax) = 1/8 £ PmlA,) 
meVy 
where Vj is the set of neighbor points of point j, of which 
point i is a member as shown in Figure 3.2. Taking the 


partial derivative, we find 


0 0 
————- 95(\1) = 1/8 ————Pj(\}) 
OP3 (A) dP; (Aj) 
= 1/8 (3.14a) 
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and similarly, 


0 


© 


Q4(A2) (3,14b) 


dP; (Aq) 
Substituting (3.14) into (3.13) leads to 
3 


ao E P5°Q3 = 1/8 EF P5(Aq) 
i‘*l’ jevy jeVi 
= Q3(A}1) G5) 
Substituting (3.15) into (3.l2a), results in 


dC 


il 


OPj (A) 


2034 0\1) 
Similarly, the second component is 


dC 
————- = 20;()\92) 
dP; (A2) 


In summary, the gradient of the criterion, C, is 


dC aC 
VC = 





dP; (A) dP3(A2) 


VC = leas (az), 204 (2) | [Ref. 18] (3.16) 
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Figure 3.2: Set of pixels Vy and V5 


An efficient method called the steepest ascent 
technique will be utilized to maximize the criterion. This 
technique begins with an initial probability, p,{0), 
i = 1,...,N for each pixel and iteratively adjusts the 
probability vector Pj to converge to a local maximum of 


criterion (3.2). This is achieved by defining a sequence 


p, (2) 


as: 
pyr) = pal) ett pounce (3a 
where ee is a positive step size, the vector Ge; Ga is the 
gradient of the function to be maximized, i.e., 
cg; '*) = VC 

dC dC 
= ee, eee for the two class case 
dP; (1) IP; (AQ) 
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(2) 


and PROJ iS edepLojveee lon, Operator that insures that p; ‘*) 


ZcectilaeaeprobabiLlityevector. [|Ref. 17] 


Based on this technique, the iteration of the 


(o) 


initial probabilities Pj is defined as 


dC 
By) = Py") (dq) + of) prog!) —__— 
9Pi (Aj) 
(241) (2) af 
Pi Oppears Vp jee p 2) prog. *) = 
dP3 (A?) (3.18) 
where 9 i%) 1s a step size which will be developed later. 


A method discussed by J. B. Rosen which maximizes a 
function while satisfying a constraint or constraints is 
called the gradient projection method [Ref. 19]. The 


constraint for this case is 


BE? (as + BaD) = ] (cor 19) 


and 


Qy(A1) + QY(A2) = 1 


205 (A]) + 203 (42) = 2 (320) 


but (3.20) is the summation of the components of the 


Mmaavent of criterion C, (3.11) and (3.16), 


dC dC 
ee eee + eee =e 
dP; (A 1) dP3 (A2) (35 2:1.) 
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The projection of the gradient at pont Pi lonetnemeroaem 
convex region, i.e., the constraint (3.19), is defined as 


L 
2 


PROJ*G; = Gj - 21 (Skv) 


Eat 


(3.229 


where PROJ is the projection operator, L is the number of 
classes, G is the gradient vector, [G1,G9,...,G2], and vw is 
[l,l,...,l]. [Ref. 14] This is shown graphically in Fig@ee 


3.3. For the two class case, 


dC dC 
Gl 2 eee) cae v= ee 
dP5 (Aj) dP; (Az) 
and the projection of the gradient (3.22) is a vector with 


two components. Substituting (3.16) into (3.22), we find 


(2) ac aC aC 
PROJ ——— | = 20;(A) -0.5|——— _ + ———— 
dP; (1) dP3(X}1) Py (AQ) 
(34.239) 
nD) dC dC dC 
PROJ * ———— | = 20;(A) -0.5|———- «+ 2~——— 
9Ps (9) 8P4 (AQ) 9Ps (AD) 


—_ 


31 is normally kept 


During each iteration, the step size oj 
constant and is the largest possible value such that after 
each iteration, the probabilities, Pj's, remain within the 
constraint of Py(Az) + Py(Agq) = 1,Pq (AR) > 0.0, k = 1,2 for 


ah sea. 
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However, for the two class case, the step size can 
be computed for each pixel. Changing the step size each 
time will provide for a faster convergence rate to the 
maximum criterion. Examples of this convergence will be 
shown later. 


L) 


The maximum value of the step size, o | is found by 


maximizing the (+1) iteration of (3.24a) and (3.24b), i.e., 


set p; (£t+1)(),) = 1 and pa 4b) = 1, respectively. This 


will produce two values for the step size, 
Ieee! (Ny) cucu (2) (204 (091) = 1) 
Mees”) (0a) = 03°") (209603) - 1) 


Bee te By) 
ea 
203(A1) - 1 


and, 
=e Pee (208015) - 1) 


eae eo) 
2Qi(42) - 1 


p, (2) = 


Substituting (3.19) and (3.20) to get 


eee et 
‘ ea Oa) 


The step size must be positive, therefore, 
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PROJ*G 


AT PUA) + p(A2) = A 


dio 1 p(y) 


Figure 3.3: Projection of the Gradient, G, 
on the constraint 
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a Pa fF) ¢Q1) 
———— 9 tO ( ky) > 0.5 
¢ 203(A1) - 1 
o;' ) = ¢ (37e2 5 ) 
P; | (a1) 
a, FO, CX, ee 0.5 
ames O70 a ) 


In the algorithm which was used in this thesis, both the 
rate of convergence to the criterion and the number of 


pixels assigned to each class was controlled by setting the 


step size to the following values, 


(2) 
“1 ? imax 
a (2) (3267) 
a2 p. pine Os XD) <- 0.5 
imax 


pede Oy xX) 2 0.5 


where a} and az are constants whose values are less than 
cone. The values of a) and az are weighting factors which 
will bias an image to a class, i} or i2, and will influence 
the convergence rate of the criterion. 

Figures 3.4(a to c) show the change in the criterion 
as the number or iterations increases for a cell image which 
was studied in the noted reference. Each figure represents 
the three cases, a1] = a9, aj < a2, and aj > az, with the 
parameter FACT = 1.0 in all cases. These figures show that 
by increasing the weighting factors, the rate of convergence 
will increase and these factors, a} and a4, will also 
control where the criterion will converge. Thus, the 


control of the relaxation process can be done. The 
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Variations of the criterion, C, 
with the iteration number for 
various values of Alphal and 


Alpha2 [Ref. 20] 
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an image at each iteration. Smoothing is defined as the 
elimination of a small region or regions of one class within 
a much larger region of the opposite class. As -the 
magnitude of each factor, a} and az increases, the smoothing 
effect decreases. This will be demonstrated in the next 
chapter. Also, the ratio of a; and ag controls the bias of 
a class. Earlier, the parameter FACT was set equal to one. 
The reason for this is shown in Figure 3.5. The effect of 
this parameter on the value of the criterion and the 
convergence rate of the criterion is seen to be minimal 
(Refs. 15, 20] 

A major capability of this process is to 
automatically select a threshold value. This is a key task 
in region segmentation. It is important in image processing 
to select an adequate threshold for extracting objects from 
their background. In the ideal case, the histogram will 
have a deep and sharp valley between two peaks representing 
the object and the background. In a real picture, however, 
it is sometimes difficult to detect the valley bottom, 
especially when the valley is flat and broad, imbued with 
noise, or when the peaks have extremely unequal heights 
producing no discernible valley. [Ref. 21] In the case 
where the histogram has a flat and broad valley, a threshold 
selected too low creates an object (target) which maybe 


larger than it actually is, or if the threshold is selected 


53 


too large, most of the actual target maybe segmented into 
the background. 

In the next chapter, it will be shown that as the 
number of iterations increases, the peaks in the histogram 
will move farther apart, and the average brightness will 
increase. When the peaks are far apart, the mean value of 
the original image or the segmented image can be used as the 
threshold value. [Ref. 15] This is why the gradient 
relaxation is advantageous as compared to other methods in 


segmenting infrared images. 
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Figure 3.5: Variation of the criterion, sevice 
iteration mumber for 3 values of FACT 
[Ref. 14] 
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IV. ANALYSIS OF IMAGES 


A. IMAGES UNDER ANALYSIS 

The gradient relaxation segmentation method discussed in 
the previous chapter was demonstrated on several infrared 
images. Ten images were used to evaluate the performance of 
this segmentation technique. The first image is a still 
photo of a ship with low contrast (poor visibility), see 
Figure 4.l(a). The other nine images were obtained from an 
uncooled focal plane infrared sensor (Figures 4.l(b) - (j)). 
The sensor was placed on a platform on which the sensor was 
rotated to simulate the situation of a rotating missile. 
This is why the targets are seen at different viewing 
angles. The images were recorded on video disc. Using the 
EYECOM digitizer, individual frames were extracted from the 
video disk. The video disc contained approximately 20 
minutes of video data of several ships in various contrasts. 
The scenes contained a wide variation of noise within the 
images. Instead of attempting to analyze all of the frames 
(approximately 64,000 frames), it was decided to select 
images which were representative of most of the frames and 
Situations depicted on the video disc. The purpose is to 
determine how effective the relaxation segmentation method 


is for these images. 


DD 





Figure 4.l({a): A ship with low coneeacre: 


Three criteria were used in the selection of the images: 


Find images where the target stands out from the 
background and is not degraded significantly by noise. 
The images which met this criteria are Figures 4.l(a), 
(c) /-and. aCaoe 

Find target near or within part or all of the 
background noise with an intensity level near the 
intensity level of the target. This is seen in 
Figures 4.l(b), (e), (f£), and (gq). 

Collect a series of frames as the target 1s rotating, 
showing how the noise changes from frame to frame. 
The series selected includes targets near noise of 
similar intensity (see Figures 4.l(g), (1), and (agge 
It also includes a target which because of the noise 
is fragmented into several objects, to the point where 


the target itself appears to be background noise (see 
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Figure 4.l(c) A sailboat 
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A large ship 
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Figure 4.l(e): First in a series of six images 
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(Ship A) 





Figure 4.1(f£): Second in a series of six images (Ship B) 
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Figure. 4:1(g) 
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Third in a series of six images 
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Figure 4.1(h): Fourth in a series of six images (Ship D) 
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Figure 4.1(h)). The intention is to see if the target 
can be segmented from the noise background well enough 
to be able to detect it as a target. 
These cases obviously do not account for all situations, but 
are representative of the noisy infrared images which were 
available for this study. 

The targeted object was then extracted from the original 
512 by 512 image to form a smaller 64 by 256 image which 
requires much less time to process. Figures 4.2(a) - (j) 
depicts each of these images with their associated 
histograms. 

Noise in the images come from various sources, either 
natural or the sensor. Noise sources include glare off the 
surface of the water, atmospheric interference, such as 
scattering and attenuation of the cloud and haze. Thermal 
noise is introduced since the senso* is uncooled. 
Transmission noise was introduced whe 1e image was 
recorded onto the video disc and when it is -gitized using 
the EYECOM digitizing system. 

The COMTAL VISION ONE/20 Image Processing System was 
used to display the images and to produce the associated 
histogram. COMTAL VISION ONE/20 is a complete image 
processing system with built-in interactive processing and 
control capabilities. The system produces high spatial 


resolution video images over a range of 256 gray levels. 
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(b) Medium-size ship from 
Figure 4.1(b) 
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Figure 4.2: Original 64 X 256 images extracted from Figure 4.1 
images with their gray-level histogram 
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(d) Large ship from Figure 
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(Figure 4.2 continued) 
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(£) Ship B from Figure 4.1(f) 
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Ship A from Figure 4.l(e) 9 
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(Figure 4.2 continued) 
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(j) Ship F FSi Figure 4.1(j) 
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(Figure 4.2 continued) 


ae 


The distribution of the pixels over the gray level range is 
completed by the COMTAL processor in the following manner. 
The processor counts all occurrences of each gray level in 
the image. This count (the total number of pixels at each 
gray level) is divided by the highest count and then 
multiplied by 256. This number is subtracted by 1 to yield 
the distribution of that gray level in the figures. The 
highest normalized count is always 255. [Ref. 22] 

The points in the original histograms were not 
connected. To provide a better feeling for the shape of the 
histogram, it was decided to connect those points which 
presented a general outline of the gray level distribution. 
The points selected are generally the highest point in a 
selected neighboring group of points. 

The histograms of each of these images generally shows 
the distribution between the background and the target. In 
Figures 4.2(a) and (e) - (j), it is possible to see a 
separation between the peak background level and the peak 
target level. However, in each of these cases it would be 
difficult to select a threshold value which could be used to 
perform an effective segmentation as discussed in Chapter 
II. By using the gradient relaxation technique, the problem 
of determining a critical threshold value is easy. 

The selection of the weighing factors, Alphal and 


Alpha2, and the number of iterations necessary to perform 
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the segmentation 1S very important, as mentioned in the 
previous chapter. The selection of these parameters is 
influenced by the detected size of the segmented target, the 
needed accuracy of the object outline, and separation of the 
gray level peaks. It also determines how quickly the 
histogram of the segmented image reaches its widest 
separation of the gray peak levels. This was also 
demonstrated in the last chapter. The following parameters 


were used in performing the experiments on the segmented 


images: 
Alphal: The weighing factor on pixels with gray 
levels greater than the mean. 
Alpha2: The weighing factor on pixels with gray 
levels less than the mean. 
Iter: The number of iterations of the 


relaxation routine. 


Threshold (THD): The threshold value is used to determine 
which pixels will be part of the labeled 
region. Two values were selected in 
each image. The first value of 220 was 
chosen because it is assumed that the 
higher intensities are part of the 
target. The second value chosen is the 
mean gray level intensity of the 
original image. 


Region: The total number of labeled regions. A 
labeled region is a grouping of pixels 
with intensities greater than the 
threshold, THD. 


Area: The number of pixels in the largest 
labeled region. 


Perimeter: The number of pixels along the boundary 
of the largest labeled region. 


1 


Shape: This is a measure of the relationship 
between the area and the perimeter of 
the largest labeled region. It is equal 
to 

Shape = 2*Area/Perimeter 
The shape is small for narrow objects. 
The shape is large for rounded objects. 
B. APPLICATIONS OF THE GRADIENT RELAXATION ROUTINE 
The relaxation routine was applied to each of the images 
shown in Figure 4.2 and are separated into ten separate 


cases. The criteria used in the analysis is as follows: 


1. Are the regions uniform and homogeneous with respect 
to a gray level? 


2. Do the regions contain gaps (holes), and if so, can 
successive iterations smooth the segmented region? 


3. Are the peaks in the histograms more distinct? 
4. Does the target conform to a desired shape? 
5. Is a target detected? 


6. Can the detected object be used in the classification 
process? 


The general format of the experiment entailed applying 
different values of the values Alphal and Alpha2 to the 
images for several iterations and to observe the effect on 
the original images. The values were subjectively chosen to 
test for the cases when Alphal = Alpha2, Alphal < Alpha2, 
and Alphal > Alpha2. The maximum number of iterations 
selected was based on the theoretical results shown in 
Figure 3.4 (Chapter III). These figures consistently showed 
that the criterion was saturated after eight or more 
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tte Yapelon S.. Using more iterations would not have 
significantly improved the segmentation. 

Each case includes a discussion on the effect of the 
algorithm on that image. A figure of the segmented image 
and the corresponding histogram are shown. Also included 
is a table summarizing the change in the area, perimeter, 
and the shape of the segmented region(s) for the different 
settings of the weighing factors, threshold, and number of 
iterations. 

1. Ship in Low Contrast (Figure 4.2(a)) 

The number of iterations is important in determining 
the peak gray level separation of the background and the 
target. Figures 4.3(a)-(d) shows how each iteration 
increases this separation. In the original image, the 
separation is approximately 45 levels; after on iteration it 
is almost 135 levels, after four iterations it is almost 
225, and after eight iterations, the separation is 
approximately 250 levels. 

Four cases involving different Alphal, Alpha2 
parameters and number of iterations were applied to the 
image of Figure 4.2(a). Results of this application are 
seen in Figure 4.3 and Table 4.1. These parameters 
determine the form and gray level intensity of the segmented 
scene. Setting the value of Alphal 2 Alpha2 increases the 


apparent size of the target. This is seen in Figures 
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Figure 4.3: Results of relaxation segmentation on ship with 
low contrast 
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TAB Gs 4 2. 
QUANTITATIVE RESULTS OF SHIP WITH LOW CONTRAST 








ALPHA1 ALPHA2 ITER THD REGION AREA PERIM SHAPE 
oe 3 O%.3 1 220 i 808 134 12.06 
Oi Os3 il 82 1 15067 607 49.65 
On 3 0.3 Z 220 1 904 eS 13.80 
0:3 0.3 2 82 I 913 1b 13.94 
0.3 0.3 4 220 1 900 eS 13.74 
Oe.3 Oe 4 82 a 905 al 3a 82 
0.3 O23 8 220 i 896 1c 13.68 
0. 3 0.3 8 82 is 899 et LB 3 
0.6 0.2 2 220 ik 912 ior ESao2 
0.6 02 2 82 4 1438 219 3.1.3 
0.6 OZ 8 220 IL fie 2 146 16.74 
0.6 Oi 8 82 ili 1222 146 16.74 
OQ. 2 0.6 Z 220 Ht 720 Ze 11.90 
0.2 0.6 Z 82 iL v7 27 ire6d 
0.2 0.6 8 220 iL 541 109 9.93 
Oe 2 0.6 8 82 i 547 109 10.04 
Orel 0.4 2 220 le 657 120 10.95 
0.1 0.4 Z 82 i 912 lise M3592 
Oi L 0.4 8 220 1 467 96 O27 3 
Oe L 0.4 8 82 1 506 102 9.92 

MEAN = 82 


4.3(a)-(f£). The resultant image looks more like a tank, not 
like a ship. By increasing the number of iterations, the 
region grows larger as defined by the area. However, in the 
cases (Figures 4.3(e)-(j)) where Alphal < Alpha2, the 
segmented region appears to be closer to the true size in 
the original image, and the region gets smaller as the 
number of iterations increase. All of the regions in each 
image are uniform and there are no holes within the regions. 
The peaks in the histogram are widely separated and 
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distinct. Results in the table shown that the shape becomes 
more clearly defined with more iterations. The table also 
shows that the mean is a reasonable value to use as a 
threshold. It is evident from the result that this type of 
scene does allow the relaxation routine to detect a target 
and would allow for the possible classification of the 
target if the proper weighing factors are selected. 
2. Medium-size Ship (Figure 4.2(b)) 

Results of this experiment are seen in Figure 4.4 
and Table 4.2. This 1S an image which clearly shows the 
effect of the number of iterations imposed on establishing 
well defined peaks. Figures 4.4(g)-(j) display the effects 
on the same image with one, two, four, and eight iterations. 
After one iteration, a valley between the peaks is better 
defined than the original histogram, and after eight 
iterations the separation is near a maximum. 

The weighing factors have a tremendous affect on the 
segmented regions. Figures 4.4(a)-(d) show that if Alphal 2 
Alpha2 the region increases in area and the target cannot be 
detected. In the case where Alphal < Alpha2 the target is 
detectable. By increasing the number of iterations, the 
segmented region develops into a form which can be neither 
detected as a ship nor classified as a ship as was seen in 
the result of the first case. Fewer iterations also 


produce more segmented regions (Table 4.2) which are small. 
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TABLE 4.2 
QUANTITATIVE RESULTS OF MEDIUM-SIZE SHIP 


ALPHA1 ALPHA2 ITER THD 
O..3 0.3 2 220 
0.3 0.3 Z 128 
O.3 0.3 8 220 
Or3 0.3 8 128 
0.6 OZ Z 220 
0.6 0.2 2 128 
0.6 OQ. 2 8 220 
0.6 0.2 8 128 
Oe 2 0.6 z 220 
0.2 0.6 2 128 
Oi: 2 0.6 8 220 
Os.2 0.6 8 128 
OL 0.2 it 220 
Oe 1 Dis 2 1 128 
Or G2 2 220 
OF | 0.2 2 128 
OF. 1 0.4 4 220 
Oh) 0.4 4 128 
0.1 0.4 8 220 
eel 0.4 8 128 

MEAN = 128 
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gray level intensities to the target and separates it from 
the adjacent noise. 
3. Sailboat (Figure 4.2(c)) 

The image is a black hot inverted infrared image. 
For the relaxation routine to work properly, the object to 
be segmented must be lighter than the background. 
Therefore, the image must be inverted first. Figures 
4.5(a)-(d) show results of segmenting this image. This image 
is similar to the first case (ship of low contrast) in that 
it provides for a detectable target and as seen in Figures 
4.5(c)-(d), it could be classified as a sailboat. This is 
more readily observed in Figure 4.5(c). This case shows 
that increasing the number of iterations does not 
necessarily decrease the size of the region as was seen in 
earlier cases (Table 4.3). The images are uniform and 
homogeneous and contain no holes; peaks are distinct, sharp, 
and widely separated. 

4. Large Ship (Figure 4.2(d)) 

This image is a good example of an object in a noisy 
background which can be segmented into an image which is 
both detectable and can be classified. Figures 4.6(a)-(d) 
depict the effect of relaxation on this image. The best 
results are seen in Figures 4.6(a) and (b) where the gray 


level peaks are clearly defined and widely separable. These 
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TABLE 4.3 
QUANTITATIVE RESULTS ON SAILBOAT 





ALPHAl ALPHA2 ITER THD REGION AREA PERIM SHAPE 
0.6 Oe 2 220 1 2309 526 8.78 
0.6 0.2 2 108 1 2724 296 18.41 
0.6 je g 220 1 2883 301 19.16 
0.6 O2 8 108 1 2883 301 19.16 
Oral 0.4 2 220 1 1051 382 5.50 
ep 0.4 2 108 1 2042 234 17245 
a 0.4 8 220 i 1488 366 8.13 
Ojeal 0.4 8 108 i 1801 Ze 16.60 

MEAN = 108 


results also allow for the easy selection of a threshold 
value. This case, and the previous cases, have demonstrated 
that the selection of a threshold to determine the area and 
the size of the region can be chosen as the mean value of 
the original image without significantly changing the 
measured parameters. The images are uniform and homogeneous 
after eight iterations in each case. Gaps are seen in the 
First iteration (Figure 4.6(a)), but are filled in @iter 
eight iterations. 
5. Series of Frames of Single Ship 
a. Ship A (Figure 4.2(e)) 

This is the first of a series of six images 
(Figures 4.2(e)-(j) which depicts a ship at various 
Orientations as the camera is rotating. This scene clearly 


shows the separation between the sky, sea, and target. The 
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Figure 4.6: Results of relaxation segmentation on large ship 
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TABLE 4.4 
QUANTITATIVE RESULTS ON LARGE SHIP 








ALPHAl ALPHA2 ITER THD REGION AREA PERIM SHAPE 
0.6 0.2 2 220 i 3170 430 14.74 
0.6 Oe 2 2 lee 1 3450 332 20.78 
0.6 0.2 8 220 1 3673 308 23.85 
0.6 One 8 dei q 3679 307 2397 
0. 1 0.4 2 220 1 248 139 3.57 
0.1 0.4 2 foe 3 2924 294 19.89 
aaa 0.4 8 220 1 1687 534 6.32 
DL 0.4 8 111 2 2569 260 19.76 

MEAN = 111 


histogram shows three distinct peaks in the gray levels of 
Figure 4.2(e). Figure 4.7(a)-(e) shows the effect of the 
segmentation on this image. This case demonstrates how a 
high threshold and few iterations will segment image into 
several regions. When Alphal 2 Alpha2, the target and the 
sky merge into one region after only two iterations. This 
prevents the detection and classification of the target. 
The situation becomes worse after eight iterations. 

In the case where Alphal < Alpha2 (Figures 
4.7(d)-(e)), the target is detectable after two iterations, 
but increasing the number of iterations creates the same 
result as the situation mentioned above; the sky and the 
target merge into one region. In this case, (see Figure 
4.2(e)) the peak associated with the sky and the object are 
greater than the mean, therefore these pixels were grouped 


together. This explains why these two areas were merged 
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96 


ALPHA] ALPHA2 
O..3 0.3 
QO. 0 3 
0.3 Wo 
O..3 O53 
0.6 OP 
0.6 OER 4 
0.6 0.2 
0.6 0.2 
0.4 Ord 
0.4 Ogu 
0.4 Oi 
0.4 Oo) 
Oi eg2 0.6 
0.2 0.6 
Oe 0.6 
OpnZ 0.6 
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MEAN 120 
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If the pixels associated with the sky had been 
the target would have been grouped into 


permitting the detection and possible 
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the desired shape of a ship does not 


be ShipaB (FProure (47 

This is the second image in the series. The sky 
is to the left, the white region to the immediate left of 
the target is caused by glare, and the sea is to the right 
of the target. This effect 1S again due to the rotation of 
the sensor. Of the series of images seen in Figure 4.8(a)- 
(e), Figure 4.8(e) permits for the detection of a target and 
its orientation. The object cannot be classified in any of 
the cases. After eight iterations, the glare and the ship 
merge into one region as would be expected based on the two 
class segmentation scheme. This is a good example of how 
noise of similar intensity near or contained within the 
target can become merged as one region. This reduces the 
ability to classify the target. Quantitative results are 
shown in Table 4.6. 

Cur Ship ee (higure 422 0o0») 

This image is similar to the previous case in 
that the background immediately surrounding the object has 
an intensity closely matching the object of interest. 
Figure 4.9 shows results of applying the relaxation 
segmentation technique. Figures 4.9(d) and (e) shows the 
cases that a target may be located in the area, or that a 


tremendous amount of glare from light reflected off the 
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Figure 4.8: Results of relaxation segmentation on Ship B 


ee 





(a) Alphal 39 Al phageawcs 





(b) Alphal "Gye pina =o 





(c) Alphal = .4, Alpha2 = .1 








(d) Alphal =-s2y-AbphaZze— 6 





Jaeea OG, | 


(e) Alphal = .1, Alpha2 = .4 





Figure 4.9: Results of relaxation segmentation on Ship C 
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TABLE 4.6 
QUANTITATIVE RESULTS OF SHIP B 








ALPHA1L ALPHA2 ITER THD REGION AREA PERIM SHAPE 
Oi 023 2 220 é 2324 476 Dia © 
Ores OAs 2 144 iE 3583 382 Sa) © 
O23 On 8 220 1 S509 367 Ro 6 
Om O23 8 144 1 Soo 366 os OZ 
0.6 Or 2 220 1 3568 406 ie OG 
0.6 Ga2Z 2 144 iL 3856 373 20.68 
0.6 0.2 8 220 il 4087 349 23.42 
0.6 OneZ 8 144 1 4091 Siow 2 Spon 
0.4 Oren 2 220 1 55.15 438 15.14 
0.4 Ore 2 144 i 3864 368 Ze 0€ 
0.4 Oma 8 220 1 4199 385 Zoo 
0.4 Gee 8 144 1 4284 384 Dera 
GZ 0.6 2 220 2 1407 402 700 
0.2 0.6 2 144 1 Bo L9 Saal Loe 
ie 0.6 8 220 i 3064 352 Lo a64 
eZ 0.6 8 144 1 3064 3hZ 19.64 
Ol 0.4 2 220 2 787 333 Ae 13 
G1 0.4 2 144 1 Bao 382 oe 
iL 0.4 8 220 1 2647 590) este 16 
Ore 0.4 8 144 1 2895 3200 18.04 

MEAN = 144 


the ocean surface (see Figure 4.2(g)). These last two cases 
clearly show that glare can have a degrading effect on the 
segmentation of the image of interest. Quantitative results 
are shown in Table 4.7. 
d. Ship D (Figure 4.2(h)) 

This is a case where noise in the image can 
cause the object of interest to be obscured. Attempts to 
segment this image were unsuccessful (see Figure 4.10). By 
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TABLE Jan 
QUANTITATIVE RESULTS OF SHIP C 








ALPHA1 ALPHA2 ITER THD REGION AREA PERIM SHAPE 
Ors 0 2 220 i Z021 483 8.3m 
‘ 02s 2 174 iy 2466 432 11.42 
Ors O33 8 20 1 Zane 370 13 a8 
Gees O73 8 174 1 2505 368 13293 
0.6 Oe2 Z 220 1 2495 462 10.80 
0.6 Oey 2 174 2 2917 402 14 .3i 
0.6 0.2 8 220 1 3265 425 15 22g 
0.6 UZ 8 174 it 3265 425 15.36 
0.4 Gen 2 220 i 2466 481 10225 
0.4 0.1 2 174 Jt 2944 431 13.66 
0.4 Oc 8 220 1 3365 437 15.40 
0.4 Deel 8 174 i 3462 426 l672ZeS 
0.2 0.6 2 220 3 oo 450 6.28 
G22 0.6 2 174 1 2133 403 10. 58 
0.2 Ure 8 220 1 1824 334 10.92 
Oie2 0.6 8 174 iE 1826 335 1Os7 
ek 0.4 2 220 2 967 431 4.49 
Gat 0.4 2 174 1 2081 431 9.66 
Ue 0.4 8 220 1 L5G, 349 Sods 
Oc 0.4 8 174 1 1604 355 Oreos 
MEAN = 174 


increasing the number of iterations, the routine produced 
only fewer and smaller regions. Thus the algorithm could 
not provide information that an object may be within the 
frame of interest. This case clearly shows that the 
relaxation method fails in this situation. Quantitative 


results are shown in Table 4.8. 
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Iter = 2 | Iter = 8 





(a) Alphal = .2, Alpha2 = .6 





Figure 4.10: Segmentation of Ship D. 


TABLE 4.8 
QUANTITATIVE RESULTS OF SHIP D 


ALPHA1L ALPHA2 ITER THD REGION AREA PERIM SHAPE 
O.292 0.6 2 220 > 64 44 ere 
0.2 OTe 2 Lo 8 308 188 Sao 
0.2 0.6 8 220 2 ihe. 39 3.84 
Ee 0.6 8 153 2 WS oy 3.84 
Om 0.4 2 220 5 Do 47 iO 
PL 0.4 2 no 8 293 178 oE2o 
0.1 0.4 8 220 1 45 45 2.00 
1 0.4 8 oe il 62 36 3.44 

MEAN = 153 
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e. Ship © (Figure 422700) 

This is the fifth image in the series for the 
same ship. Figure 4.11 and Table 4.9 shows the results of 
segmentation. This is a Situation where a possible object 
may be in this frame. This case demonstrates that by 
increasing the number of iterations, the segmented region is 
more clearly defined by eliminating the noise near the 
Ob ect of interest. Also, more iterations reduces the 
number of segmented regions. The target is detected in the 
case. However, it does not allow for the classification of 


this target. 





(a) Alphal = .2, Alpha2 = 26 





(DPA pha la Alpha2 = .4 


Figure 4.11: Segmentation of Ship E 


104 


TABLE 4.9 
OUAN T Ghee RESUIOES OF SSHIP E 


ALPHA1 ALPHA2 ITER THD REGION AREA PERIM SHAPE 
OFZ 0.6 2 220 5 950 241 7.88 
Or 2 0.6 Za 189 D 1087 25 8.66 
Or 2 0.6 8 220 il 857 190 02 
ONG A 0.6 8 189 il 857 ra0 2) UY 
Or 0.4 2 220 4 853 278 6.14 
On 0.4 2 189 5 1042 249 B37 
Or 0.4 8 220 1 699 152 97.20 
Ore 0.4 8 189 J. ee] 161 oF.0 3 

MEAN = 189 


f. Ship F (Figure 4.2(3)) 

The final image which was analyzed shows results 
(Figure 4.12 and Table 4.10) which are similar to hose seen 
in Figures 4.8 and 4.9. The glare which dominates the left 
side of the ship merges into the same region of the ship 
after only two iterations. It makes classification 
impossible and greatly reduces the possibility of detection. 
This case also demonstrates how several iterations can 
reduce the size of the segmented region. This case also 
demonstrates that by having a lower threshold value (i.e., 
the mean), there are fewer segmented regions (1 versus 6, or 
1 versus 3), thus enabling an observer to focus on the one 


large region. 
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ALPHA1 ALPHA2 
DZ CaS 
OreZ O=5 
Ore 0.6 
Or Oe 
Oak 0.4 
Or 0.4 
Ore): 0.4 
ber 0.4 

MEAN = 171 
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QUANTITATIVE RESULTS OF SHIP F 


AREA 
eZee 
2411 
203 
2056 


794 
2341 
7 nS 
eon 


PERIM 
318 
437 
361 
S0nl 





280 
430 
324 
So 


SHAPE 

T.oo 
Liss 
ll .3s 
1 des 9 


5. 67 
10282 
10.332 
10.86 


C. SUMMARY OF RESULTS 

The results show that for these cases, where the target 
has a high gray level and contains noise due to the 
environment and the sensor, it is best to have Alphal < 
Alpha2. This reduces the chance that noise which ahs gray 
levels greater than the mean will be included in the desired 
segmented region. Care must betaken to select appropriate 
values for Alphal and Alpha2, otherwise, the region will 
become so small that the object of interest is not 
classifiable. The target may still be detectable however. 
The result will provide the orientation of the target. 

The process works well on an image which is similar to 
Figure 4.2(a). The peaks in the histogram are clearly 
defined and are sharp, not bell-shaped as in the case of the 
noisy images (Figures 4.1(b)-(j)). The noisy images can be 
segmented and generally identifiable if the target to be 
segmented iS approximately ten percent or more of the frame 
of interest (Figures 4.2(b)-(e)). However, if the object 
occupies less than three percent of the image plane (Figure 
4.2(h)), it is difficult or impossible, as in this case, to 
segment it by this method. 

The segmented region in all cases was uniform, 
homogeneous, and any holes within the region were eliminated 
after several iterations. These are all desirable 


properties of in a segmentation routine. In summary, all 


Oy, 


but one of the cases (Figure 4.2(h)) providedm@onmesene 
detection of a possible target, and four of the test images 
(Figures 4.3(a), (b), (c), and (d)) could be used as an 


input to a classification system. 
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V. CONCLUSION 


Image segmentation is a critical step in the image 
analysis and pattern recognition process. Errors which 
occur at this step may propagate through additional stages 
of a pattern recognition system producing an incorrect 
description of the scene. The gradient relaxation technique 
is an iterative probability adjustment technique that can be 
used for segmentation. It takes advantage of both 
'‘parallel' and the 'sequential' processing methods. The 
relaxation approach itself has two major advantages: 1) the 
classification decisions become better informed as the 
analysis proceeds, and 2) the method can use probabilistic 
classifications rather than making firm decisions 
immediately. 

The approach is conducive to the segmentation problem of 
noisy infrared images having unimodal distributions. Noise 
near or within the target will be filtered out because each 
pixel's probability classification is adjusted based on the 
probabilistic classification of its neighbors. The gradient 
relaxation technique maximizes the gray level intensity of 
the target allowing for easier detection. The weighting 
factors must be chosen carefully. These factors are 
critical in determining the rate of convergence (length of 


Og 


time to maximize the intensities), the extent that noise is 
eliminated from the image and the shape of the segmented 
region. The technique is still a subjective process and the 
ability of the observer to set the proper values of these 
factors 1S important. 

The relaxation method is an ideal technique for region 
extraction because of its ability to sharpen the peaks in 
the histogram, create homogeneous and uniform regions, and 
the detected target conforms well to its original shape, 
1.@€., a Ship. This method is not suitable for edge 
detection of objects in noisy infrared images. The noise 
causes gaps in the edges at places where the transition 
between regions are not abrupt. Additional edges may be 
detected at points that are not part of the region 
boundaries. 

Noisy images are primarily unimodal making the selection 
of @ threshold difirculec. This analysis showed that the 
threshold can be easily selected as the mean gray level 
intensity of the image. This allows for precious 
computational time to be spent for segmentation or other 
image processing, instead of being spent to search for a 
threshold for additional image analysis. 

The technique is unable to separate noise of similar 
gray-level intensity near or within the target. This 


introduces errors into the image segmentation result, making 


aie 


classification of the target difficult, if not impossible. 
The technique fails to segment targets which are not 
contiguous (i.e., broken up by the noise). The intended 
target either is segmented into several small regions or (if 
the intensity level of the noise is near that of the target) 
becomes part of the noise. This makes detection and 
classification of the target impossible. 

This technique could be implemented in hardware as part 
of a signal processor. By implementing the technique as 
part of the processor, the requirements for an infrared 
sensor could be reduced. Possible requirements which would 
be reduced or eliminated include signal-to-noise ratio, 
detectivity and cooling requirements of the sensor, the 
weight and power for the system would possibly be reduced. 
Money saved in the cost of the sensor could be used to 
enhance the computing capabilities of the signal processor. 
Possible applications are missiles, remotely piloted 
vehicles (RPV's), aircraft, and remote sensors aboard 
spacecraft. 

In summary, the gradient relaxation technique is a 
viable method to use in uncooled infrared sensors to detect 
targets. The ability of the technique to eliminate or 
reduce noise of intensity less than the target, thus 
enhancing the target and to provide for detection has been 


shown. The technique could possibly be used as one of the 


ie 


inputs of a classification process (i.e., shape matching) or 
classification system, but only for those images where the 
intensity of the target is greater than that of the noise, 
or where the target has large spatial separation from the 


noise of similar intensity. 
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APPENDIX: EXPERIMENTAL PROCEDURE 


The infrared images used in this analysis were obtained 
from an infrared uncooled focal plane array sensor. The 
images were then recorded and stored on a video disc. Using 
the EYECOM digitizing system, individual frames were 
extracted from the video disc. The EYECOM system creates an 
image file of 640 blocks of 512 bytes. This file must be 
reduced to 512 blocks of 512 bytes in order to be displayed 
on the COMTAL image processing system. This file was 
further reduced to 64 blocks of 256 bytes to reduce the 
processing time. 

The measurements made in Chapter IV of the area and 
perimeter were obtained by calling subroutines in the 
Subroutine Package for Image Data Enhancement and 
Recognition (SPIDER) image processing package. The routines 
which were used are: 

1. CLAB - The routine assigns labels (serial numbers) 
each segmented region. Each pixel in a region is 
assigned a label. This routine produces a labeled 
image. 


2. AREA1 - The routine counts the number of pixels within 
every region in a labeled image. 


3. PRMT1 - This routine meaSures the perimeter of every 
region in a labeled image. [Ref. 23] 
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